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•• The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

• Failure to reply vyrithin the set or extended period for reply v^ll, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)13 Responsive to communiGation(s) filed on 26 May 2005 . 
2a)n This action is FINAL. 2b)^ This action is non-final. 

3) n Since this application is in condition for allowance except for formal nnatters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-22 is/are pending in the application. 
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5) 0 Claim(s) is/are allowed. 

6) S Claim(s) 1-22 is/are rejected. 
?)□ Claim(s) is/are objected to. 
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Application Papers 
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Replacement drawing sheet(s) including the con-ection is required If the drawing(s) is objected to. See 37 CFR 1.121(d). 
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1 .□ Certified copies of the priority documents have been received, 

2. n Certified copies of the priority documents have been received in Application No. . 
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applicafion from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received, 
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DETAILED ACTION 
Continued Examination Under 37 CFR L114 

1. A request for continued examination under 37 CFR 1. 1 14, including the fee set forth in 37 CFR 1. 17(e), 
was filed in this application after final rejection. Since this application is eligible for continued examination under 
37 CFR L 1 14, and the fee set forth in 37 CFR 1. 17(e) has been timely paid, the finahty of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Apphcant's submission filed on 27 April 2005 has been entered. 

Response to Arguments 

2. Applicant's arguments filed on 29 March 2005 have been fully considered. A response to these arguments 
is provided below. 

Prior Art Rejections 

Summary of Argument : 

Applicant has amended the independent claims and provided arguments directed to the newly amended subject 
matter. 

Examiner *s Response : 

The new limitations will be addressed in the below rejection. 

Claim Rejections - 55 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the 
rejections imder this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1-3, 10-12, 19, and 20 are rejected under 35 U.S.C. 102(b) as being anticipated by Bender et al. 
(USPN 5,657,402). 

With regard to claim 10, which is representative of claim 1, Bender discloses a sensor for generating input 
data and a processor module coupled to the sensor (Bender col. 3 lines 14-17). 

Bender further discloses selecting a first frame of data as a template frame and capturing a second frame of 
data using the EO system (Bender col. 12 hues 13-17 in conjunction with Fig, 6). Bender discloses registering 
frame 201 with frame 202. In this particular situation, frames 201 and 202 from Bender are analogous to the 
claimed second frame and template frame, respectively. Bender further discloses capturing these frames with a 
video camera (Bender col. 24 hnes 46-48). This qualifies as an EO system as recited in the claim (see paragraph 
[0005] of the apphcant's disclosure). 
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Bender further discloses correlating at least a portion of the second frame with the template frame to 
generate a shift vector (Bender col. 1 1 line 66 - col. 12 line 7). Bender discloses a process for determining partial 
derivatives Ix, ly and It These partial derivate terms, which are used in the subsequent interpolation operation, 
qualify as the shift vectors recited in the claim in that they correspond to a difference between the frames (aka 
shift) and they are vectors by definition. It follows that the determination of the shift vector as disclosed in Bender 
is analogous to the claimed process of correlating the frames. Indeed, the reference describes determining how far 
apart two frames are (col. 12 lines 10-12). 

Bender further discloses registering the second frame with the template frame by: 

(A) Interpolating the second frame using the shift vector (Bender col. 1 1 line 19 - col. 12 line 47: The 
reference describes an affine transformation (col. 1 1 lines 23-36) which uses the shift vector (col. 1 1 line 65 - col. 
12). Interpolation is inherent in the affine transformation operation.); and 

(B) Resampling by spatially oversampling at least a portion of the second frame by a factor greater than 
one to produce a registered fi^e (Bender col. 12 line 63 - col. 13 line 6: The reference describes a resampling 
operation of the frame which is being warped. This operation is disclosed to "fill in these spaces with information" 
(i.e oversampling). The reference describes that this operation uses bilinear interpolation — which is well known in 
the art to sample at a factor greater than one). 

Bender further discloses resampling the template frame (Bender col. 14 lines 30 - 50). In the cited passage 
Bender discloses a situation where a frame 25 1 is being warped (registered) with a frame 252. In this case, frame 
252 is analogous to the claimed template frame and after the warping is done, frame 251 is registered with the 
template frame. It follows that the template frame 252 is warped with the scale of frame 253. Consequently, 
Bender discloses resampling the template frame. 

Bender further discloses combining the resampled template frame and the registered frame to generate an 
. averaged frame (col. 17 lines 24-27 with element 414 of Figure 12). The temporal median filter disclosed in 
Bender generates an averaged frame of the resampled template frame and the registered frame. 

Bender further discloses selecting another frame as an updated template frame to which a subsequently 
captured frame of data is registered (Bender, Figure 14 in conjunction with col. 19 lines 47-62 and the above 
argument provided in the 'response to arguments section*, which will not be repeated herein). 

With regard to claims 19 and 20, Bender discloses a predetermined number of intervals as 1. This is seen 
in Figure 14, where each new frame serves as a template frame for the subsequently-captured frame. 

With regard to claim 1 1, which is representative of claim 2, Bender further discloses using bilinear 
interpolation in the step of registering a second frame with a template finame (col. 13 lines 4-6). The bilinear 
interpolation disclosed in Bender occurs in the process of resampling a portion of the second frame. This process 
is part of the step of registering the second fi^e. 

With regard to claim 12, which is representative of claim 3, Bender discloses adding motion to a line of 
sight of the EO system using a conunanded line of sight pattern or a random pattern to generate multiple frames of 
data (col. 23 lines 1-7 and lines 55-65). The alteration of the field of view among the images in a sequence as 
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disclosed in Bender is analogous to adding motion to a line of sight as recited in the claim. Bender does not disclose 
whether the field of view of the images in a sequence is altered according to a pattern or whether it is altered 
randomly. However, it is inherent that Bender's alteration of the field of view has to be performed either randomly 
or according to a pattern, seeing that no other options exist Consequently, Bender teaches all of the Umitations of 
the claim. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set 
forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 6, 7, 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bender as apphed to 
claims 1 and 10 above, and further in view of Hanna et al. (USPN 6,269,175). The arguments as to the relevance of 
Bender as apphed in paragraph 2 above are incorporated herein. 

With regard to claim 15, which is representative of claim 6, Bender fails to expressly disclose resampling 
the averaged frame. Hanna, however, discloses a "compositing process" which is analogous to the averaged frame 
generating process (or combining process) recited in the claim, and further discloses resampling the averaged frame 
data after the compositing process (Hanna col. 12 lines 13-49). The filling of unfilled pixels disclosed in Bender 
qualifies as the claimed image resampling. It would have been obvious to one reasonably skilled in the art at the 
time of the invention to modify Bender's image processing apparatus by resampling the previously averaged frame 
data as taught by Hanna. Such a modification would have allowed for a synthesized output image with a higher 
resolution. 

With regard to claim 16, which is representative of claim 7, Hanna discloses using an upsampled frame for 
the purpose of filling in previously unfilled pixels (Hanna col 12 lines 46-49). Hanna further discloses that the 
upsampled frame was interpolated using a bilinear interpolation method (Hanna col. 1 1 lines 55-57), 

7. Claims 4 and 13 are rejeaed under 35 U.S.C. 103(a) as being unpatentable over Bender as apphed to 
claims 1 and 10 above, and further in view of Komiya et al, (USPN 6,205,259). The arguments as to the relevance 
of Bender as apphed in paragraph 2 above are incorporated herein. 

With regard to claim 13, which is representative of claim 4, Bender discloses determining an averaged 
frame, but fails to expressly disclose spatially filtering the averaged frame in order to enhance the edges. Komiya, 
however, discloses an image synthesizing circuit (which determines an averaged firame as recited in the claim) 
which is connected to the input of an edge emphasizing circuit (Komiya col. 24 lines 41-50 with Figure 41). It 
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would have been obvious to one reasonably skilled in the art at the time of the invention to modify Bender's image 
processing apparatus by adding an edge enhancer for the averaged frame data as taught by Komiya. Such a 
modification would have allowed for an output image which contained uniformly enhanced edges. 

8. Claims 8, 9, 17 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bender as apphed to 
claims 1 and 10 above, and further in view of Van Ackere et al. (USPN 6,047,028), The arguments as to the 
relevance of Bender as apphed in paragraph 2 above are incorporated herein. 

With regard to claim 17, which is representative of claim 8, Bender fails to expressly disclose temporally 
filtering a first fi:ame to generate the template frame. Van Ackere, however discloses temporally filtering a first 
frame in order to generate the template frame (Van Ackere abstraa). It woxdd have been obvious to one reasonably 
skilled in the art at the time of the invention to modify Bender's image processing apparatus by temporally filtering 
an input to generate the template frame as taught by Van Ackere. Such a modification would have allowd for a 
template (reference) image with less noise (Van Ackere abstract). 

With regard to claim 18, which is representative of claim 9, Bender further discloses a resampling 
operation which utihzes bilinear interpolation (Bender col. 13 lines 1-6). 

9. Claims 5 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bender as apphed to 
claims 1 and 10 above, and further in view of Chen (USPN 6,556,704). The arguments as to the relevance of 
Bender as apphed in paragraph 2 above are incorporated herein. 

With regard to claim 14, which is representative of claim 5, Bender discloses determining an average frame, but fails 
to expressly disclose utilizing a histogram to change its pixel depth. Chen, however, discloses changing pixel depth 
on the basis of a histogram (Chen col. 4 lines 1-13). It would have been obvious to one reasonably skilled in the art 
at the time of the invention to modify Bender's image processing system by including a method for utihzing a 
histogram in order to change pixel depth. Such a modification would have allowed for an output image that had 
more clearly imaged background and foreground regions and consequently was more pleasant to look at (Chen col. 
4hnes M3). 

10. Claims 21 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Bergen (USPN 
6,208,765) in view of Martins et al. (USPN 6,438,275). 

Regarding claim 22, which is representative of claim 21, Bergen discloses a sensor for generating input data, 
and a processor module coupled to the sensor (Bergen Figure 1). 

Bergen further discloses selecting a first frame of image data as a template frame (Bergen col. 3 lines 16-18). 
Bergen further discloses capturing a second frame of image data (Bergen col. 3 lines 18-19). 
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Bergen further discloses aligning the second frame of image data with the template frame of image data by 
correcting line of sight deviations therebetween (Bergen col. 3 lines 1-5 & 18-20: The reference discloses aligning 
images that have a "slightly different perspective.")- 

Bergen ftirther discloses spatially oversampling at least a portion of the aligned second fi^me of image data by a 
factor greater than one (Bergen col. 3 lines 25-32). 

Bergen further discloses integrating the spatially oversampled portion of the aligned second frame of image data 
with image data of the template fi^ame (Bergen col. 3 lines 33-45). 

Bergen fails to expressly disclose repeating the above steps for subsequently captured frames of data repeating 
the above steps to process subsequently captured frames of data into a continuous video stream. Bergen is directed 
to the processing of video frames to produce a single enhanced image, but does not disclose repeating this operation 
to make multiple enhanced frames. Martins, on the other hand, teaches that individual video frames can be 
enhanced by increasing their resolution (Martins col. 1 lines 29-3 1). It would have been obvious to one reasonably 
skilled in the art at the time of the invention to modify Bergen's image resolution enhancement system by applying 
it to multiple frames as taught by Martins. Such a modification would have allowed for the enhancement — by way 
of increased resolution — of a continuous video stream (or moving image), and would have resulted in an improved 
quahty video image (Bergen col. 1 lines 29-30). 

Conclusion 

1 1 . Any inquiry concerning this conununication or earlier conununications from the examiner should be 
directed to Patrick L Edwards whose telephone number is (571) 272-7390. The examiner can normally be reached 
on 8:30am - 5:00pm M-F. 

If attempts to reach the examiner by telephone are unsuccessftil, the examiner's supervisor, Joe Mancuso 
can be reached on (571) 272-7695, The fax phone number for the organization where this apphcation or proceeding 
is assigned is 703-872-9306. 

Information regarding the status of an apphcation may be obtained from the Patent Application Information 
Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR 
or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto,gov. Should you have questions on access to the 
Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




